The present invention relates to a method and an apparatus for processing image data, and, more particularly, to an image data processing method which generates primary color data representing primary color components from complementary color data representing complementary color components.
FIG. 1 is a block diagram showing the configuration of a prior art image sensing device 100 which uses a CCD image sensor 1, and FIG. 2 is a plane view showing an example of a mosaic color filter attached to the CCD image sensor 1.
The CCD image sensor 1 has a plurality of light-receiving pixels, a plurality of vertical shift registers and usually a horizontal shift register. The light-receiving pixels are arranged in a matrix form on the light-receiving surface at regular intervals and produce and store information charges corresponding to the image of each sensed object. The vertical shift registers are arranged to correspond to the columns of the light-receiving pixels and sequentially shift the information charges stored in the light-receiving pixels, in the vertical direction. The horizontal shift register is arranged on the output side of the vertical shift registers and receives the information charges output from the vertical shift registers, and then transfers the information charges row by row. This allows the horizontal shift register to output an image signal I0 which changes a voltage value in accordance with the information charges stored in the light-receiving pixels.
An analog processing circuit 2 performs a process, such as sampling and holding or level clamping, on the image signal I0 input from the CCD image sensor 1 to produce an image signal I1 which conforms to a predetermined format. For example, in the sample and hold process, only an image signal having a certain signal level is extracted from the image signal I0 having reset levels and signal levels which are alternately repeated in synchronism with the output operation of the CCD image sensor 1. In the level clamping process, the black reference level set at the end of the horizontal scanning period of the image signal I0 is clamped to a predetermined level every horizontal scanning period. An A/D conversion circuit 3 quantizes the image signal I1 received from the analog processing circuit 2 in synchronism with the operation of the analog processing circuit 2, i.e., the output operation of the CCD image sensor 1, to generate image data D which represents the information with a digital value corresponding to each light-receiving pixel of the CCD image sensor 1.
A digital processing circuit 4 performs a process, such as color distribution or a matrix operation, on the image data D received from the A/D conversion circuit 3 and generates luminance data Y and color difference data U and V. For example, in the color distribution process, the image data D is separated in accordance with the color arrangement of a color filter attached to each light-receiving surface of the CCD image sensor 1, generating a plurality of color component data. Further, in the matrix operation process, primary color data corresponding to the three primary colors of light are generated from the individually separated color component data, and are then combined at a predetermined ratio, thereby generating the color difference data.
A driver 5 responds to various timing signals from a timing control circuit 6 and supplies a multi-phase drive clock to each shift register of the CCD image sensor 1. For example, a 4-phase vertical transfer clock xcfx86v is supplied to the vertical shift registers, and a 2-phase horizontal transfer clock xcfx86h is supplied to the horizontal shift register. In accordance with a reference clock having a given period, the timing control circuit 6 produces a vertical timing signal, which determines the vertical scan timing of the CCD image sensor 1, and a horizontal timing signal, which determines the horizontal scan timing, and supplies the timing signals to the driver 5. At the same time, the timing control circuit 6 supplies a timing clock CT for synchronizing the operation of each circuit 2, 3, 4 with the output operation to the analog processing circuit 2, the A/D conversion circuit 3 and the digital processing circuit 4.
In performing color image sensing, attaching a color filter for color distribution to the light-receiving surface allows the individual light-receiving pixels of the CCD 1 to be associated with predetermined color components. A stripe type color filter has a plurality of segments each of which is connected in the vertical direction and a mosaic type color filter which has a plurality of segments associated with every light-receiving pixel. For example, the mosaic type color filter, as shown in FIG. 2, is split into a plurality of segments corresponding to each pixel of the light-receiving section of the CCD image sensor 1 and color components of Ye (yellow), Cy (cyan), W (white) and G (green) are cyclically assigned to each segment. In the example of FIG. 2, the W and G components are alternately arranged in odd rows and the Ye and Cy components are alternately arranged in even rows. For an image signal obtained from the CCD image sensor 1 to which such color filter is attached, the W and G components are repeated when reading even rows and the Ye and Cy components are repeated when reading odd rows.
FIG. 3 is a block diagram showing the configuration of the digital signal processing section 4, and FIG. 4 is a timing diagram for describing the operation of the processing section 4. FIG. 4 corresponds to the case where the mosaic type color filter shown in FIG. 2 is attached to the CCD 1.
A color distribution circuit 11 separates the image data D in which each color component continues in the arrangement order of each segment of the color filter. The distribution circuit 11 then generates color component data C[Ye], C[Cy], C[G] and C[W]. For the image data D input from the A/D conversion circuit 3, as shown in FIG. 4, the G and W components continue alternately in an operation of reading odd rows (ODD) and the Ye and Cy components continue alternately in an operation of reading even rows (EVEN). Accordingly, the color distribution circuit 11 retains at least one row of the image data D to allow the output of all the color component data C[Ye], C[Cy], C[G] and C[W] at the time of reading each row. Specifically, during reading of an odd row, the color distribution circuit 11 separates the image data D for the odd row and outputs the color component data C[G] and C[W]. At the same time, the color distribution circuit 11 separates the image data D for the previously read even row and outputs the color component data C[Ye] and C[Cy]. Further, this scheme causes the color component data C[Ye], C[Cy], C[G] and C[W] to be output intermittently at the time of outputting the image data D serially. The intermittent portions of the color component data are however interpolated by outputting the same color component data twice in succession.
A color calculation circuit 12 performs a color computation process according to, for example, the following equations on the color component data C[Ye], C[Cy], C[G] and C[W] input from the color distribution circuit 11, to generate primary color data P[R], P[G] and P[B] corresponding to the three primary colors (R: red, G: green and B: blue) of light.
Yexe2x88x92G=R
Cyxe2x88x92G=B
G=G
A white balance control circuit 13 assigns specific gains set to each of the primary color data P[R], P[G] and P[B] input from the color calculation circuit 12 to adjust the balance of each color. In other words, in the white balance control circuit 13, because this adjustment compensates for differences in the sensitivities of the light-receiving pixels of the CCD image sensor which depend on each color component, the gains of the primary color data P[R], P[G] and P[B] are individually set to improve the color reproduction of a reproduced image.
A color difference matrix circuit 14 generates color difference data U and V from the primary color data P[R], P[G] and P[B] input from the white balance control circuit 13. The color difference matrix circuit 14 combines the respective primary color data P[R], P[G] and P[B] at a ratio of 3:6:1 to generate luminance information. Then, the color difference matrix circuit 14 subtracts the luminance information from the primary color data P[B] corresponding to the B component to generate the color difference data U. Further, the color difference matrix circuit 14 subtracts the luminance information from the primary color data P[R] corresponding to the R component to generate the color difference data V.
A luminance calculation circuit 15 combines the four color components included in the image data D provided to the color distribution circuit 11 to generate the luminance data Y. That is, assume each component of Ye, Cy, G and W is combined. It can be seen that
Ye+Cy+G+W=(B+G)+(R+G)+G+(R+G+B)=2R+4G+2B
This allows the luminance data Y in which the R, G and B components are combined at a ratio of 1:2:1 to be obtained. While a luminance signal is produced by combining the R, G and B components at a ratio of 3:6:1 according to the NTSC standards, the luminance signal produced by combining the components at a ratio close to this ratio does not cause a practical problem.
An aperture circuit 16 enhances a specific frequency component included in the luminance data Y to generate aperture data, and adds the aperture data Y to the luminance data Y. In other words, to enhance the outline of the image of a sensed object, the aperture circuit 16 performs a filtering process on the image data D to generate aperture data so that the frequency component of one fourth the sampling frequency, which is used to obtain the image data D from the image signal Y, is enhanced. The luminance data Y generated in this manner is supplied to an external display device or recording device together with the color difference data U and V.
Because the R and B components are generated in the color computation process by the color calculation circuit 12 by subtracting the G component from the Ye and Cy components, respectively, the R or B component may show a negative value according to the unevenness of spectral characteristics of the color filter. For example, for light in which the G component is strong and the R or B component is weak, the Ye or Cy component has a slightly higher value than the G component and the R or B component should show a positive value close to xe2x80x9c0xe2x80x9d. However, if the Ye or Cy filter is not be transparent to desired light and the Ye or Cy component can be obtained only by a lower value than the G component, the R or B component has a negative value as a result of the color computation process. Such negative value cannot be originally obtained as a color component, and produces a false signal, thereby causing the image quality of the reproduced image to deteriorate.
It is an object of the present invention to provide an image signal processing apparatus which does not generate a false signal even if unevenness of spectral sensitivities of the color filter occurs.
In one aspect of the present invention, a method is provided that processes first to third complementary color data representing complementary colors of the three primary colors of light and produces first to third primary color data representing the three primary colors of light. The method includes the steps of multiplying said first complementary color data and said second complementary color data to generate a first product, multiplying said first complementary color data and said third complementary color data to generate a second product, multiplying said second complementary color data and said third complementary color data to generate a third product, calculating a square root of the first product to produce a first root as the first primary color data, calculating a square root of the second product to produce a second root as the second primary color data, and calculating a square root of the third product to produce a third root as the third primary color data.
In another aspect of the present invention, an apparatus is provided that processes image data comprising first to third complementary color data representing complementary colors for the three primary colors of light and generates first to third primary color data representing the three primary colors of light. The apparatus includes a distribution circuit for distributing said image data into the first to third complementary color data. A multiplication circuit multiplies the first and second complementary color data to produce a first product, the second and third complementary color data to produce a second product, and the first and third complementary color data to produce a third product. An extraction circuit extracts square roots of said first to third products and produces first to third roots as said first to third primary color data.